Primal-dual splitting algorithm for solving inclusions with 
mixtures of composite, Lipschitzian, and parallel-sum type 

monotone operators* 

—^ . Patrick L. Combettes^ and Jean-Christophe Pesquet^ 

(N 

{3JQ; ^UPMC Universite Paris 06 

^ ' Laboratoire Jacques-Louis Lions - UMR CNRS 7598 

75005 Paris, France 
C"~~- I [pIcOmath . jussieu . f r| 

f) , ^Universite Paris-Est 

Q ; Laboratoire d'Informatique Gaspard Monge - UMR CNRS 8049 

^ ■ 77454 Marne la Vallee Cedex 2, France 

■y ■ |iean-ch.ristophe.pesquet@univ-paris-est .frl 
C^ 



(N 
> 

oo 

O ■ Abstract 

C^^ ' We propose a primal-dual splitting algorithm for solving monotone inclusions involving a mixture 

^^ , of sums, linear compositions, and parallel sums of set-valued and Lipschitzian operators. An 
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can be processed individually via explicit steps, while the set-valued operators are processed 
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1 Introduction 

Duality theory occupies a central place in classical optimization [121 IMl IMl SDl HI]- Since the 
mid 1960s it has expanded in various directions, e.g., variational inequalities [2| 1171 ^^ 1231 [26| I34j. 
minimax and saddle point problems [STl ESI 1321 ES] j and, from a more global perspective, monotone 
inclusions [3 [9l |10l [161 ED EZl |38]. In the present paper, we propose an algorithm for solving the 
following structured duality framework for monotone inclusions that encompasses the above cited 
works. In this formulation, we denote hy BDD the parallel sum of two set-valued operators B and 
D (see (j2.5p ). This operation plays a central role in convex analysis and monotone operator theory. 
In particular, BDD can be seen as a regularization of B by D, and D is naturally connected to 
addition through duality since {B + D)~^ = B~^DD^^. It is also strongly related to the infimal 
convolution of functions through subdifferentials. We refer the reader to [SI [281 ESI (Ml US] and the 
references therein for background on the parallel sum. 

Problem 1.1 Let ?^ be a real Hilbert space, let z € Ti, let m he a, strictly positive integer, let 
A: Ti ^ 2"- he maximally monotone, and let C: "H — )• "H be monotone and /x-Lipschitzian for some 
// € ]0, -|-oo[. For every i € {1, . . . , m}, let Qi he a real Hilbert space, let r^ G Gi, let Bi: Gi ^ 2^» be 
maximally monotone, let L>j : t/j — >• 2^* be monotone and such that D^ is z^j-Lipschitzian, for some 
Vi G ]0, -|-oo[, and suppose that Li/. T-L ^ Qi is a nonzero bounded linear operator. The problem is to 
solve the primal inclusion 

m 
find X &n svid\t\isX z & Ax + ^L*[{BiUDi){LiX -ri)) +Cx, (1.1) 

i=X 
together with the dual inclusion 



{2 — V™ L*W G Ax + Cx 
(V^ E |1,... ,m|) Vi G {BiUDi)(LiX - ri 



(1.2) 



Problem 11.1 1 captures and extends various existing problem formulations. Here are some examples 
that illustrate its versatility and the breadth of its scope. 

Example 1.2 In Problem II. II set 

m = l, C7:x^0, and Di : y ^ J ' ^ ^ ~ °' (1.3) 

1^0, if y /O. 

Then we recover a duality framework investigated in [lOl [HI [371 EH] , namely (we drop the subscript 
'1' for brevity), 

, , {z(^Ax + L*{B(Lx-r)] , , 

find (x,U GT^ee such that { , \ ^ '{ , (1.4 

[-r G -L(^-i(z - L*U)) + B-%. 

Example 1.3 In Example 11.21 let ^ = 7^, r = z = 0, and L = Id . Then we obtain the duality 
setting of OEI], i.e.. 



Q^ Ax^Bx 

OG -A~-^{-u)^B-^u. 



find (^, u) G 7^ e ?^ such that <( ^ _ ^__^ _ _^_ (1.5) 



The special case of variational inequalities was first treated in 

Example 1.4 In Example 11.21 let A and B be the subdifferentials of lower semicontinuous convex 
functions f : 7i ^ ]— oo,+oo] and g: Q ^ ]— oo,+oo], respectively. Then, under suitable constraint 
qualification, we obtain the classical Fenchel-Rockafellar duality framework [IQ], i.e., 

minimize f{x) + g{Lx — r) — {x \ z) 

""^^ (1.6) 

minimize f*{z — L*v) + g*{v) + {v \r). 

Example 1.5 In Problem [TTH set C: x H- 0, z = 0, and (Vi € {1, ... , m}) Qi = Ti, ri = 0, Lj = Id , 

and Di = p^ Id, where pi € ]0,+oo[. Then it follows from [HI Proposition 23.6(ii)] that, for every 

i € {1, . . . , m}, BiDDi = (Id —JpisJ/Pi = ^'Bi is the Yosida approximation of index pi of Bi. Thus, 

(jl.ip reduces to 

m 

find X en such that e Ax + ^ P'BiX. (1.7) 

«=i 

This primal problem is investigated in [13,1 Section 6.3]. In the case when m = 1, we obtain the 
primal-dual problem (we drop the subscript '1' for brevity) 

{0 £ Ax + ^Bx 
(1-8) 
e -A-\-u) + B-^u + pu 

investigated in [9]. 

Example 1.6 In Problem ll.il set m = 1, Qi = Q, Li = L, z = 0, and ri = 0, and let A and Bi be the 
subdifferentials of lower semicontinuous convex functions / : 7^ — )• ]— oo, -|-cxj] and g: Q ^ ]— co, +00], 
respectively. In addition, let C be the gradient of a differentiable convex function /i : 7^ ^ R, and 
let D be the subdifferential of a lower semicontinuous strongly convex function £: t/ — > ]— oo,-|-oo]. 
Then, under suitable constraint qualification, (jl.ip assumes the form of the minimization problem 

minimize f{x) + {gDi){Lx) + h{x), (1-9) 

which can be rewritten as 

minimize f{x) + h{x)+g{y)+i{Lx — y). (1-10) 

In the special case when h = 0, Q = "H, L = Id , and £ is a quadratic coupling function, such 
formulations have been investigated in [H HI O [121 IIS] • 

Example 1.7 In Problem [TTTl set m = 1, Gi = Ti, Li = Id , Bi = D^^ : x ^ {0}, and z = ri = 0. 
Then ([l.ip yields the inclusion G Ax + Cx studied in [l5] , where an algorithm using explicit steps 
for C was proposed. 

Example 1.8 In Problem 1 1.1[ set A: x >-^ {0} and C = Id. Furthermore, for every i G {1, . . . ,m}, 
let Bi be the subdifferential of a lower semicontinuous convex function gi: Qi — )• ]— oo,-|-oo] and 



let D^ : y i— )■ {0}. Then, under suitable constraint qualification, we obtain the primal-dual pair 
considered in [14J, namely 

m I 

minimize \ gi{LiX — Vi) -\ — ||x — z|| (l-H) 



and 

m 



... 1 

minimize — 

VieQl,...,VmeGm. 2 



z 

i=l 



E ^>^ 



+ Y.^9:{vi) + {v,\n)). (1.12) 



i=l 



Example 1.9 The special case of Problem 11.11 in which 

*' ^ y~ ' (1.13) 

0, if y / 

yields the primal-dual pair 

m 

find x(^H such that ^ G ^ L* {Bi{LiX - rj)) (1.14) 

j=i 
and 

(3x E ?^)(V^ G {1, . . . , m}) W G 5i(LiX - ri). 
This framework is considered in [10^ Theorem 3.8]. 

Conceptually, the primal problem (jl.ip could be recast in the form of (|1.14p . namely 

m 

find X en such that ^ G ^ L*[Ei{LiX - rj)), (1.16) 

1=0 

where 

Qo = n, Eo = A + C, Lo = Id, ro = 0, and (Vi G {l,...,m}) ^^ = BiODi. (1.17) 

In turn, one could contemplate the possibility of using the primal-dual algorithm proposed in [ 1U\ 
Theorem 3.8] to solve Problem ll.il However, this algorithm requires the computation of the resolvents 
of the operators A + C and {B^ + D^ )i<i<m, which are usually intractable. Thus, for numerical 
purposes. Problem 11.11 cannot be reduced to Example 11.91 Let us stress that, even in the instance 
of the simple inclusion G Ax + Cx, it is precisely the objective of the forward-backward splitting 
algorithm and its variants [H [151 EOl Sll SH] to circumvent the computation of the resolvent of vl + C, 
as would impose a naive application of the proximal point algorithm 



The goal of this paper is to propose a fully split algorithm for solving Problem 11.11 that employs 
the operators A, (Lj)i<j<m, (-Bj)i<j<m, {Di)i<i<m, and C separately. An important feature of the 
algorithm is to activate the single- valued operators (ij)i<j<m, {Dj^ )i<i<m, and C through explicit 
steps. In addition, it exhibits a highly parallel structure which allows for the simultaneous activation 
of the operators involved. This new splitting method goes significantly beyond the state-of-the-art, 
which is limited to specific subclasses of Problem 11.11 

In Section[21 we briefiy set our notation. The new splitting method is proposed in Section^ where 
we also prove its convergence. The special case of minimization problems is discussed in Section HI 



2 Notation and background 

Our notation is standard. We refer the reader to O US] for background on convex analysis and 
monotone operator theory. Hereafter, /C is a real Hilbert space. 

We denote the scalar product of a Hilbert space by (• | •) and the associated norm by || • ||. The 
symbols ^ and — ?• denote respectively weak and strong convergence. Moreover, ^i © • • • ® Gm is the 
Hilbert direct sum of the Hilbert spaces {Gi)i<i<m in Problem ll.il i.e., their product space equipped 
with the norm {yi)i<i<m ^ \/YlT=i lly«P- ■'^^^ every i G {1, . . . , m}, let Tj be a mapping from Qi to 
some set TZ. Then 

mm m 

i=l i=l i=l 

Let M: /C — > 2^^ be a set-valued operator. We denote by ranM = |u G /C I (3x G /C) n G Mx\ 
the range of M, by dom M = |x G /C | Mx 7^ 0} its domain, by zer M = |2;G/C|0g Mx^ its set of 
zeros, by graM = {(j;,u) G /C x /C | u G Mx} its graph, and by M~^ its inverse, i.e., the set-valued 
operator with graph |(ii, x) G /C x /C | u G Mx}. The resolvent of M is 

Jm = (Id +M)-\ (2.2) 

where Id denotes the identity operator on /C. Moreover, M is monotone if 

(V(x,u) G graM) (V(y,t;) G graM) {x-y\u-v)>Q, (2.3) 

and maximally so if there exists no monotone operator M: /C ^ 2 such that graAf C graM ^ 
graM. We say that M is uniformly monotone at x G domM if there exists an increasing function 
(p: [0, +oo[ — )• [0, +00] that vanishes only at such that 

(Vu G Mx)(V(2/,v) G graM) {x - y \u - v) > 4){\\x - y\\). (2.4) 

The parallel sum of two set-valued operators Mi and M2 from /C to 2 is 

Ml nMa = {M^^ + M:^^)-^. (2.5) 

We denote by Tq{}C) the class of lower semicontinuous convex functions tp: IC ^- ]— 00, +00] such 
that dom 99 = {x G /C | y?(x) < -|-oo} 7^ 0. Now let cp G Tq{}C). The conjugate of if is the function 
if* G Tq{IC) defined hy if* : u t-^ sup^£;(^((x | u) — v^(x)), and the sub differential of (p is the maximally 
monotone operator 

dip: IC ^ 2'^ : X ^ {u e IC \ {yy e IC) {y - x \ u) + ip{x) < (fiiy)} (2.6) 

with inverse given by 

{d^)-'=d^*. (2.7) 

Moreover, for every x G /C, (/?+ |[x — -11^/2 possesses a unique minimizer, which is denoted by prox x. 
We have 

prox^ = Jd^- (2.8) 



We say that 93 is z^-strongly convex for some u € ]0,+oo[ if 99 — v\ ■ |p/2 is convex, and that (/? is 
uniformly convex at x G domc^ if there exists an increasing function 0: [0,+oo[ -^ [0, +00] that 
vanishes only at such that 

(Vy G dom(^)(Va G ]0, 1[) ipiax + (1 - a)y) + a(l - a)^{\x - y||) < a^ix) + (1 - Oi)^){y). (2.9) 

The infimal convolution of two functions 991 and 992 from /C to ] — 00, +00] is 

ip\Ukp2: /C ^ [-00, +00] : X i-> inf (991(2/) + (^2(2; - y)). (2-10) 

Finally, let S* be a convex subset of /C. The strong relative interior of 5, i.e., the set of points 
X £ S such that the cone generated by —x + S" is a closed vector subspace of JC, is denoted by sri5, 
and the relative interior of S, i.e., the set of points x € S such that the cone generated by —x + 5 is 
a vector subspace of /C, is denoted by riS*. 

3 Main result 

Our main result is the following theorem, which presents our new splitting algorithm and describes 
its asymptotic behavior. 



Theorem 3.1 In Prohlem M.R suppose that 



A + Y,L*{BinDi){U- 
1=1 



-n) + C 



(3.1) 



Let (ai^„)„(=N; (&i,n)neN) o-nd {ci^n)n&i be absolutely summable sequences in T-i and, for every 
i G {l,...,m}, let (a2,j,n)neN) (&2,i,n)nGN7 CLnd (c2,i,n)neN be absolutely summable sequences in Qi. 
Furthermore, set 



13 = max{/u,z/i,...,i/m} + 



Vllr-I|2 
. 2^11^*11 , 



(3.2) 



let xo G %, let {vifl, . . . ,fm,o) G Qi 
[e, (1 — e)//3], and set 



Qm, let e G ]0, l/(/9 + 1)[, let {'~in)n&\ be a sequence in 



(Vn G N) 



yi,n = Xn- ln{CXn + Ya=\ K^i,n + ai,n) 
Pl,n = J-i^A{y\,n + 7n^;) + fel,n 



For z = 1, . . . ,?7i 



y2,i,n = -yj,™ + 7n(ij2;„ - B^ ^Vi^n + a2,i,n) 
P2,i,n = J^ D-i(y2,i,n " 7n^i) + b2,i,n 
I2,i,n = P2,i,n + ln{LiPl,n - D~ P2,i,n + C2,j,n) 
Vi,n+1 = Vi,n — 2/2,4,71 + Q2,i,n- 

qi,n = Pl,n - 7n{Cpi,n + Yl'iLl L*P2,i,n + Cl,n) 

Xn+1 = Xn- yi,n + qi,n- 



(3.3) 



Then the following hold. 



(i) EngN W^n - Pl,n\? < +00 and (Vi G {1, . . . , m}) J2neN \\'^i,n - P2,i,n|P < +00. 



(ii) There exist a solution x to (jl.ip and a solution {vi, . . . ,Vm) to ()1.2p such that the following 
hold. 

(a) z - X^JLi L*vJ e Ax + Cx and (Vi E {1, . . . , m}) LjX - r j G i?~"'^W + D^^vl. 

(b) (ViG{l,...,m}) -riG-Li((A-inC'i)(z-E7=i^;W))+Sr'W+I)r'W. 

(c) Xn ^ X and pi^„ ^ 5f. 

(d) (Vi G {1, . . . ,m}) Wj^n ^ vl and p2,i,n -^ vi- 
le) Suppose that A or C is uniformly monotone at x. Then x„ — )• x and pi^n —^ x- 

(f) Suppose that, for some i E {1, . . . ,m}, B^ or D^ is uniformly monotone at vl. Then 
Vi,n -^ W and P2,i,n -> W- 



Proof. Let us first rewrite (j3.3p as 



(Vn G N) 



yi,n = Xn- ln{CXn + YT=1 ^'^^,n + ai,n) 

For i = 1, . . . ,?n, 

|_ ?/2,J,n = ■Wj.n + 7n(ij2;n " D~^Vi^n + a2,i,n) 
Pl,n = J'y„A{yi,n + InZ) + 6l,n 

For i = 1, . . . ,?TT, 

P2,i,n = J^, D-i (y2,i,n " 7n?^j) + ^2,i,n 
gi.n = Pl,n - 7n(Cpi,„ + Ya=1 ^iP2,i,n + Cl,„) 

For i = 1, . . . ,m 

[ q2,i,n = P2,i,n + ln{LiPl,n " -0^^2,4,71 + C2,j,n) 

For i = 1, . . . ,?TT, 

L ^^i,n+l = ^^j,n — y2,i,n + Q2,i,n- 



Next, let us introduce the Hilbert space 

and the operators 

M:K -^ 2^ 

{x,vi,...,Vm) ^ {-z + Ax)x{ri + B^^vi] 

and 



X (r„ + B^Vm) 



(3.4) 



Q:K ^ K 

{x,vi,...,Vm) ^ [Cx + Llvi^ VL*„^Vm,-Lix + D:^^vi,...,-LjnX + D-}vm)- 

Since the operator A and (-Bi)i<i<m are maximally monotone, so is M by [8, Propositions 
and 20.23]. In addition, [8", Propositions 23.15(ii) and 23.16] yield 

(V7G]0,+oo[)(V(x,vi,...,i'^)gK:) 

J^fM{x,Vi,...,Vm) = {.J.yA{x + -fz), J^^-i{vi -7ri),... , J^^-l(fm - Tr^)) • 



(3.5) 
(3.6) 

(3.7) 
20.22 

(3.8) 



Let us now examine the properties of Q. To this end, let {x,vi, . . . , Vm) and (y, u;i, . . . , Wm) be two 
points in /C. Using the monotonicity of the operators C and {D^ )i<i<m, we derive from ()3.7p that 

{{x,Vi,...,Vm) - iy,Wi,...,Wm) \ Qix, Vi, . . . ,U,n) - Q{y,Wi, . . . , Wm)) 

= {{x-y,vi -Wi,...,Vm -Wm) \ {Cx - Cy + L\{vi - Wi) ^ V L*^{Vm -Wm), 

- Li{x-y) + D^^^vi - L>f ^wi , . . . , -Lm{x - y) + D:^Vm - D:^Wm) ) 

m 

= {x - y \ Cx - Cy) + '^{vi-Wi\ Dr^Vi - D'r^Wi) 

i=l 
m 

+ ^{'\^-y\ ^ti'^i - ^0) - {vi - Wi I Li{x - y))) 
=1 

m 

y\Cx- Cy) + ^^{vi-Wi\ D~^Vi - Dr^Wi) 



i=l 



4 = 1 



>0. 



(3.9) 



Hence, Q is monotone. Using the triangle inequality, the Lipschitzianity assumptions, the Cauchy- 
Schwarz inequality, and ()3.2p . we obtain 

\\Q{x,Vi,... ,Vm) - Q{y,Wi,... ,Wm)\\ 

Cx - Cy, D^^vi - D^'^wi, ..., D:^Vm - D:^w„ 



< 



+ 



(in 
^ L*{vi - Wi), -Li{x -y),..., -Lm{x - y) 
i=i 

Cx - Cy, D^^vi - D^^wi, ..., D:^^Vm - D~^w. 

( X] ^«*(^* ~ ^')' ~^i(^ - y)> • • • ; -Lm{x - y) j 
^ i=i ^ 



\ 



|Cx-Cy||2 + ^||D-i^;,-D^ 



-Vir + 



j=i 



i 



2 rn 



Y,^iv^-W^) +Y,\\Liix 



i=l 



1=1 



< 



\ 



/i^llx - y||2 + '^I'fWvi - WiW^ + 
1=1 

< max{ij,,ui,...,i^m} 



\ 



m \ 2 "I 

y^ ll^ill Ibi - "^ill ) + X^ ll-^iP ll^; - y\ 

i=l ^ 2=1 



\ 



yP + E 



Wi - Wi 



i=l 



+ 



^^Eiw)(E 



\Vi - Wi 



4 = 1 



') + (Ell^^p)ll^-y!l 



{x,Vi,... ,Vm) - {y,Wi,...,Wm)\\- 

To sum up, we have shown that 

M is maximally monotone and Q is monotone and /3-Lipschitzian. 



(3.10) 
(3.11) 



Next, let us observe that 

m 

^^^{^x(^rL) z (^ Ax + ^L*{{BinDi){LiX - n)) +Cx 



1=1 



<^{3{x,Vi,...,Vm) <^1C) < 



'z(^Ax + YZiL>i + Cx 
vie{BiUDi){Lix-ri) 

G -z + Ax + Y.T=i L>i + Cx 
G n + Sf ^wi + Df ^f 1 - Lix 

<^ (3(x,fi,... ,fm) G /C) < . 

, U G r,^ + -t),^ VjYi + i^jT^ f m ^m^ 

■^ {3{x,vi,...,Vm) G/C) (0, ...,0) G (-2 + Ax) X {ri+B^\i) x ••• x (r„ + 5„^t;^) 

+ {Llvi -\ h Lm^m + Cx, D^^Vi - LiX, . . . , D^^^^Vm - LmX) 

(3.12) 



^i3{x,vi,...,Vm) G/C) (0,...,0) G (M + Q)(x,wi,...,w 
In other words, 



zer(M + Q) / 0. 



(3.13) 



Now, let us set 



(Vn G N) < 



Vn = {yi,n,y2,l,n,--- ,y2,m,n) 
Pn = (Pl,n,P2,l,n, • • • ,P2,m,n) 
Qn = {Ql,n, q2,l,n, • • • , q2,m,n) 



and 



Ctn — (ai,n; a2,l,n; • • • , 0.2,m,n) 
bn = (fel,n, ^2,l,n, • • • , ^2,m,n) 
Cn — (,Cl ji) C2 1 rt) • • • ) C2m,nj- 



(3.14) 



We first observe that our assumptions imply that 

y \\an\\ < +00, y^ ||6„|| < +CX3, and N^ |[c„|| < +cx3. 



(3.15) 



neN 



riGN 



nGf 



Furthermore, it follows from l\3.7\i . (j3.8p . and ()3.14p . that ()3.4p assumes in /C the form of the error- 
tolerant forward-backward-forward algorithm 



(Vn G N) 



Pn = JynM Vn + bn 

Qn = Pn - IniQPn + Cn) 

•^n+1 — "^n Vn ~r Qn' 



(3.16) 



(i)J It follows from (ISTTTl . (IXT3D . ([3J5]l . (fXT6]) . and [TOl Theorem 2.5(i)] that XI^sn W^n-PnW < 



+ 0O. 



(ii) It follows from |101 Theorem 2.5(ii)] that there exists x G zer(iW + Q) such that 

Xn ^^ X and p„ ^ ^. 



(3.17) 



Let us set 

In view of dMD and dMD, 



X G zer(M + Q) <^ < 



^ < 



X = {x,Vi,...,Vm)- 

'Qe-z + Ax + YJlLi L*v'i + Cx 
G ri + Sf ^UI + D^^vT - Lix 

'z-J2T=iL*vJ£Ax + Cx 



^ < 



z — X]jLi L*!!] G ^^ + Cz 
We(SinDi)(Lix-ri) 



z — X^jLi -^ji^ G Ax + Cx 
L*WeLt((SinL>i)(Lix-ri)) 



^L;;^^;^ G Ll^{{BraUDm){LmX - Vm)) 
m 

^ z e Ax + ^L*{{BiUDi){LiX - n)) + Cx 

i=l 

^ X solves (jl.ip . 
On the other hand, p.20p means that 

{vl, . . . , Iv^r) solves (!1.2p . 



(ii)(a) This follows from (fXTnU . 



(ii)(b) We derive from (1X1^]) that 



e{A + C)-^(z-Y,L)~j) and (Vi G {1, . . . ,m}) L.[x - n e {B^^ + D-^] 

^ 7=1 ^ 



Hence, 



'-<--"» fc!:,;t^:^^-"-^''^' 



Thus, 



(3.18) 



(3.19) 



(3.20) 



(3.21) 
(3.22) 



(3.23) 



(3.24) 



(ViG{l,...,m}) -ri£-Li((A'^nC-^)L-Y,^~?j^+Bi^-i + D~^vl. (3.25) 
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SK^ This follows from (IXT71) . (IXTHD . and ([MI])- 



(ii)(e) : Let us set 



(ii)(d)| This follows from i^JT^, ([XTHj) . and (1X2^ . 

(Vn G N) I ^1'" = ^" - ^- (^^" + ^7=1 ^i^i.") 



and 



P2,i,n = J^^B-^^y'i^i^ri - Inri)- 



(Vie {!,..., m})(VnGN) 

Then, in view of ()3.3p . 

(Vn G N) ||yi,„ - yi,„|[ < 7n||ai,n|I < /3~-^||ai,„|| 
and, using the nonexpansiveness of the resolvents ^ Proposition 23.7], we obtain 

(Vn € N) ||pi,„ -Pl,n|| < ||./7„A(yi,n + InZ) + 6l,„ - J^„A{yi,n + lnZ)\\ 

< Il2/l,n - yi,n|| + ||&l,n|| 

< /3"^||ai,„|| + ||6i,n||- 

Since the sequences {ai^n)nen and (6i,n)nGN are absolutely summable, it follows that 

yi,n - yi,n -^ and pi,„ - Pi,n -^ 0. 
Using the same arguments, we derive from (|3.3|) and (|3.27|) that 

(Vi G {1, . . . , m,}) 7/2,i,n - y2,i,n ^ and P2,i,n - P2,i,n ^ 0. 



On the other hand, we deduce from (ii)(a) that there exists u ^71 such that 



and that 



In addition, p.26p yields 



u € Ax and z = u + N^ I-'*Vj + Cx, 
(Vi e {1, . . . , m}) LjX - r j - -D^"^!)! G -Br^T)! 



(Vn G N) 7„ Ha;„ - pi,„) - Cx„ - ^ -^jWj,„ + z G Api,„ 



while (|3:27|) yields 

(Vi G {1, . . . , m}){\/n G N) 7^^(ui,„ - P2,i,n) + LiXn - D^'^Vi^n - Tj G B^^p2,i,n- 

Now let us set 

/w ^x /"l,n = \\Xn -Pl,n\\{e^^\\Pl,n -x\\ + lJ-\\Xn -x\\ + YaU 11^*11 ll'^J.n "WH 

(Vn G i^j < ^ _^ _ _ 

I a2,n = Lj=l(^ + '^ijlhi.n -P2,j,n|l \\P2,i,n - Vi\\- 



(3.26) 



(3.27) 



(3.28) 



(3.29) 
(3.30) 
(3.31) 

(3.32) 
(3.33) 

(3.34) 

(3.35) 
(3.36) 
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It follows from[(i)] p)(c)||(ii)(d)i ^M, and (|H:^ that 

ai,n -^ and a2,n — ^ 0. (3.37) 

Using the Cauchy-Schwarz inequality, and the Lipschitzianity and nionotonicity of C, we obtain 



(Vn G N) ai,„ + ( x„ - x 



E^:c 



''^j ^i.n. 



i=l 



> \\Xn -Pl,n\\{e ^\\Pl,n " x\\ + |lCx„ - Cx\\) + ( pi,„ - Xr. 

m \ 

^ L* {Wi - Vi^n) 



E^*c 



Vi "^i.n. 



~r \ 2^71 X 



i=l 



i=l 



E^: 



i^j "^i.n, 



i=l 



> 



> 



Pl,n - X 



Pl,n - X 



Pl,n - X 



Pl,n - X 



i=l 



Xn -Pl,n\\{£ ^\\Pl,n " x\\ + \\CXn - Cx\\) + (pi^n " X 

m \ 

7~^(X„ - pi,„) + ^ L*(IJJ - i;j,„) \ + {pi^n -Xn\Cx- CXn) 

i=l ' 

m m V 

7~^(x„ - pi,n) - E -^j*^'." ~ C'a;„ + ^ L*W + Cx\ + {x - Xn\Cx - Cx„) 

m \ 

7~^(X„ - pi,„) - ^ -^*t^j,n - C'Xn + Z-u) + {x-Xn\Cx- CXn) 
=1 ' 

y^ L*Vi^n - Cxn + z\ -u). (3.38) 

i=l ' I 



In \Xn - Pl,r. 



Now suppose that A is uniformly monotone at x. Then, in view of (|3.32p . (j3.34p . and (|3.38p . there 
exists an increasing function (pA'- [0, +oo[ -^ [0, +00] that vanishes only at such that 



(Vn G N) «!,„ + [xn- X 



i=l 



^L*{vi -Vi^n)) > 4>A{\\Pl,n -x\\). 



(3.39) 



On the other hand, it follows from (|3.36p . the Lipschitzianity of the operators [D^ )i<i<m, (|3.33p . 
()3.35p . and the monotonicity of the operators {B~ )i<i<m and {D~ )i<i<m that 



(Vn G N) Q2,n + (xn-X ^ L*{P2,i,n " Vi) 
^ i=l 

m 

- E (^n H^i,n - P2,i,n) " D^^Vi^n + D^^p2^i^n + Li{Xn " x) \ P2,i,n " W) 

i=l 
m 

= E ((7n ^(^J,n - P2,i,n) + ^i^Cn " D'^Vi^n - Ti - {LiX - Ti - DT^Vl) \ P2,i,n " W) 



i=l 



+ (Z). V2,i,n - D^ ^Vi I P2,i,n " Vi 



>0. 



(3.40) 
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Adding S^:^ and (IHriOjl yields 

(Vre G N) ai^n + a2,n + (xn-x 



i=l 



^L*(p2,i,n-i;i,n)) > M\\Pl,n-x\\). (3.41) 



It then follows from ([5:771) , |(ii)(c)|[(i)| <^Mh . and [F, Lemma 2.41(iii)] that (j)A{\\pi,n -x\\) -^ and, 
in turn, that pi^n -^ x. Hence, in view of (i) and ()3.30p . we get x„ ^ x and pi^n — >■ x. Likewise, if C 
is uniformly monotone at x, there exists an increasing function (pc ■ [0, +oo[ — > [0, +oo] that vanishes 
only at such that 



(Vn G N) ai,„ + a2,n + (xn- x 



y^ L* {P2,i,n -Vi,n)) > (t>c{\\Xn -x\\), 



i=l 



(3.42) 



and we reach the same conclusion. 

(ii)(f) Suppose that B^ is uniformly monotone at vl for some i G {1, . . . , m}. Then, proceeding 



as in (j3.40p . there exists an increasing function 0^. : [0, +oo[ — )• [0, +oo] that vanishes only at such 
that 



(Vn G N) a2,n + { Xn - x 



Z]^KP2j,r 



i=i 



m / 

- XI ( yn^^'":!'-^ ~ i^2,i,n) + LjXn - Dj^Vj^n " Tj - {LjX - Tj - DJ^V]) \ P2j,n " Vj) 
.7=1 ^ 



D, ^P2,j,n - D- ^Vj I P2J,n " Vj 



-j FZj,i 



'j ""J 1 y^,],' 



- X^ yn'^i^l-^' ~ ^2,i,n.) + LjXn - D- ^Vj^n " Tj - [LjX - Vj - D- ^Vj) \ P2,j,n 

i=i 

> (7n ^(Wi,n - P2,i,n) + LiXn - D'^Vi^n - n - [LiX - Tj - Dl^vl) \ P2,i,n " W) 

> 4>BA\\P2,i,n-nl\\)- 



Vi 



On the other hand, according to ()3.38p . 

(Vn G N) ai^n + {xn-x 
Hence, 

(Vn G N) ai^n + a2,n + ( x„ - x 



^L*{v]-vj,n))>0. 
j=i 



(3.43) 



(3.44) 



i=i 



n - Vj^n) ) > (pB,[\\P2,i,n " Vj, 



(3.45) 



By proceeding as previously, we infer that P2,i,n —^ Vi and hence, via ()3.3ip and (i) , that P2,i,n -^ Vi 
and Vi^n — ^ vi. If D~ is uniformly monotone at vl, the same arguments lead to these conclusions. D 

In the following remarks, we comment on the structure of the proposed algorithm and its relation 
to existing work. 
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Remark 3.2 Here are some observations regarding the structure of algorithm 

(i) The algorithm achieves full splitting in that each of the operators appearing in Problem 11.11 is 
used separately. 

(ii) The algorithm uses explicit steps for the single-valued operators and implicit steps for the 
set-valued operators. Since explicit steps are typically much easier to implement than implicit 
steps, the algorithm therefore exploits efficiently the properties of the operators. 

(iii) The sequences {ai^n)neN, (&i,n)nGN, and (ci,„)„gN, and, for every i £ {l,...,m}, (a2,i,„)neN, 
{i>2,i,n)nGn, and (c2,i,n)neN relax the requirement for exact evaluations of the operators over the 
course of the iterations. 

(iv) Most of the elementary steps in (|3.3p can be executed in parallel. 

(v) The update of the variable P2,i,n can also be carried out using the resolvent of Bi since [51 

Proposition 23.18] J^^^-i(y2,i,n - Inri) = y2,i,n - InTi - 7n«^^-iB,(7n^(y2,J,n " InTi)). 

Remark 3.3 Some noteworthy connections between Theorem 13.11 and existing work are the follow- 
ing. 

(i) Unlike most splitting methods, the proposed algorithm is designed to solve explicitly a dual 
problem. 

(ii) In the special case when m = 1 and Di is as in ()1.3p , the primal problem (jl.ip reduces to (we 
drop the subscript '1' for brevity) 

find xeH such that z (^ Ax + L* {B{Lx - r)) + Cx, (3.46) 

the dual problem p.2p reduces to 

find U eg such that - r £ -L{{A + Cy\z - L*v)) + B-^v, (3.47) 

and the algorithm is governed by the iteration 

yi,n = Xn- ln[CXn + L*Vn + ai,„) 

y2,n = Vn + 7n{LXn + a2,n) 

Pl,n = J-ir,A{yi,n + InZ) + 6l,n 

P2,n = J-y^B-i {y2,n " Tn?") + b2,n 

qi,n = Pl,n - ln{Cpi^n + L*P2,n + Cl,n) 

q2,n = P2,n + 7n(-^Pl,n + C2,n) 

Xn+l = Xn- yi,n + ft,n 

Vn+l =Vn- 2/2,n + q2,n- 

On the one hand, if C: x i-t' 0, we recover the primal-dual setting of [lOj and its algorithm 
f flOl Eq. (3.1)]). On the other hand, if L: x ^ 0, B: y ^ {0}, z = 0, and r = 0, (lOGjl yields 



(3.48) 
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the problem studied in [JS], and (|3.48|) without error terms and dual variables yields a primal 
algorithm proposed in that paper, namely 

yi,n — ^n ^n'^^n 

Pl,n = J^„Ayi,n .3_4gx 

qi,n = Pl,n - InCpi^n 

Xn+1 = Xn- yi,n + qi,n- 

Let us note that, even when we specialize ()3.46p to G = Ti and L = Id , there does not appear to 
exist an alternative algorithm that splits A, B, and C and uses explicit steps on the Lipschitzian 
operator C. 

(iii) When C: x H^ and, for every i G {l,...,m}, D^^ : y i— ;► {0}, we recover the primal-dual 
setting of |10^ Theorem 3.8]. However, the algorithm we obtain is different from that proposed 
in that paper, and novel. 

(iv) In general, the weak convergence results of Theorem I3.1|(ii)| cannot be improved to strong 



convergence without additional hypotheses on the operators such as those described in (ii)(e 



and (ii)(f) Indeed, in the special case when (jl.ip reduces to the problem of finding a zero of A, 
the primal component of (|3.3p reduces to the proximal point algorithm, namely (set C: x i— )• 
in dSai) 

(Vn G N) Xn+i = J^^AXn-, (3.50) 

which is known to converge weakly but not strongly [71 [25] . 



4 Minimization problems 

The proposed monotone operator splitting algorithm can be applied to a broader class of problems 
than that within the reach of existing splitting methods. It has therefore potential applications 
in the areas in which these methods have been used, e.g., partial differential equations [211 l30] . 
mechanics [22l [31], variational inequalities [H [HI |31], game theory yoj, traffic theory [20], and 
evolution equations [3]. In this section, we focus on the application of the results of Section [3] to 
convex minimization problems. 

Problem 4.1 Let 'H be a real Hilbert space, let z € 'H, let ?tt, be a strictly positive integer, let 
/ G ro(?^), and let /i: 'H — ?• M be convex and differentiable with a /i-Lipschitzian gradient for some 
^ G ]0,+cx3[. For every i G {1,... ,w,}, let Qi be a real Hilbert space, let rj G Qi^ let gi G ro(^i), 
let ii G ro(^i) be 1/fj-strongly convex, for some Vi G ]0,+oo[, and suppose that Li : Ti ^ Qi is a, 
nonzero bounded linear operator. Consider the problem 



minimize /(x) + > {giD£i){LiX — ri) + h{x) — {x \ z), (4-1) 

1=1 

and the dual problem 

z - J2 L>i J + ^ {gtiv,) + £*{v^) + {vi I u)) . (4.2) 



mmimize 



15 



The following result is an offspring of Theorem I3.1[ 
Theorem 4.2 In Prohlem\4-l\ suppose that 



(rn 
df + Y,L*{dgiDd£i){Li- 
i=l 



-ri) + \/h]. 



(4.3) 



Let {ai^n)neN, (f'i,n)neN; and {ci^n)neN be absolutely summable sequences in V. and, for every 
i € {l,...,m}, let (a2,i,n)nGN; (fe2,«,n)neN, o-^d (c2,i,n)nGN be absolutely summable sequences in Qi. 
Furthermore, set 



P = max{^,z^i,...,i/m} + 



Vll7"-I|2 
. 2^11^*11 , 



(4.4) 



let xo e %, let {vifi, ... , Vmfl) G Qi 
[e, (1 — e)//3], and set 



Qm, let e € ]0, l/(/3 + 1)[, let (7n)ngN be a sequence in 



{Wne 



yi,n = Xn- ln{^h{Xn) + Ya=1 L*i'^i,n + ai,n) 

pi^n = prox^^j(yi,„ + 7„z) + 6i,„ 
For i = 1, ... ,771 

y2,i,n = Vi^n + ln{LiXn - Vi*{vi,n) + a2,i,n) 
P2,i,n = prOX^^g. (y2,i,n " 7nfj) + b2,i,n 
q2,i,n = P2,i,n + 7n(-^iPl,n " V^*(p2,i,n) + C2,j,n) 
^^i,n+l = ^i,n — y2,i,n + <Z2,i,n- 

qi,n = Pl,n - 7n(V/l(pi,n) + YlT^l L*P2,i,n + Cl,n) 

Xn+1 = Xn- yi,n + 9l,n- 



(4.5) 



Then the following hold. 



(i) EnGNll^n -Pl.nlP < +00 0776? (Vi G {1,...,777}) X^^gp^ Huj.n -p2,i,n|P < +00. 

(ii) There exist a solution x to (|4.ip and a solution (vi, . . . , Iv^) to ()4.2p stic/7 that the following 
hold. 

(a) z - Ef=i ^*W e df{x) + V/7(Ie) and (Vi E {1, . . . , 777}) Ux - r, G %*(!}-) + V£*{vl). 

(b) x„ ^ X and pi^n -^ X. 

(c) (V-7 G {1, . . . ,777}) Vi^n -^ Vl and P2,i,n -^ vi- 

(d) Suppose that f or h is uniformly convex at x. Then Xn ^fX and pi^n -^ x- 

(e) Suppose that, for some i £ {1, . . . , m}, g* or £* is uniformly convex at vl. Then vi^n — ^ W 
and p2,i,n -^ vi. 

Proof. Let us first establish a connection between Problem 14. II and Problem 11.11 To this end, let us 
define 

A = df, C = V/7, and (Vi G {1, . . . , 777}) B, = dgi and A = d£i. (4.6) 

It is clear that g3D yields ([34]) and, using I^J}) and ([SS]), that g3]) yields ([33]) • Moreover, it follows 
from [SI Theorem 20.40] that the operators A and {Bi)Ki<_m are maximally monotone, and from [HI 
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Proposition 17.10] that C is monotone. On the other hand, for every i € {l,...,m}, it follows 
from the 1/fj-strong convexity of ii and [8] Corohary 13.33 and Theorem 18.15] that i* is Frechet 
differentiable on Qi with a fj-Lipschitzian gradient, and from (j2.7p that D~ = V£*. Altogether, we 
can apply Theorem 13.11 to obtain the existence of a point x & H such that 



z e df{x) + Y,midgiadei){LiX-n)) + Vh{x), (4.7) 

and of an m,-tuple (Ui , . . . , v^) € ^i © • • • © Gm such that 

l(Vie{i,...,m}) tiie(a9in«i)(iii-ri), 



that satisfy (i) and (ii) It remains to show that x solve ()4.ip and (vi, . . . , v^) solves ()4.2p . We first 



observe that since, for every i € {1, . . . ,m}, dom^* = ^« [SI Proposition 24.27] yields 

(Vie{l,...,m}) dgiDd£i = d{ginii). (4.9) 

On the other hand, it follows from [8, Corollary 16.38(iii) and Proposition 17.26(i)] that 

d{f + h-{-\z))=df + Vh-z. (4.10) 

As a result, we derive from (j4.7p that 

m 

Oed{f + h-{-\ z)){x) + Y,L*{d{9iOii){LiX-ri)). (4.11) 

i=l 

However, since ()4.3p and [8", Proposition 16.5(ii)] imply that 

m / m \ 

d{f + h- {■ \ z)) +Y,L*{d{gin£i)){Li- -n) (1 dif + h- {■ \ z) + Y,{gi^^i)°{Li- -ri)\, (4.12) 

it follows from (j4.1ip that 

G 9^/ + /i - (• I z) + Y.{g^nii) o (Li • -ri)") (x). (4.13) 

Thus, Fermat's rule [SI Theorem 16.2] asserts that x solves ()4.ip . Finally, to show that (t)i, . . . ,u^) 
solves ()4.2p . we first note that it follows from (|4.10p . (|2.7p . and [8, Proposition 15.2] that 

(9/ + v/i)-^ = {d{f + M)"' = 5(/ + hf = d{rnh*). (4.14) 

Likewise, (jM]) and [Bl Proposition 13.21(i)] yield 

(ViG{l,...,m}) (%n9£i)-' = 5(5^0^,)* = 9(5* + ^*). (4.15) 

Hence, combining (|4.8p . ()4.14p . and ()4.15p . we obtain 

\(ViG{l,...,m}) LiX-riGa((7*+^:)(W) 
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and therefore 



(Bxen) < 



i=l 



iLix)i<,<m e-ixLi) (d{r □ h*){z - ZT=i L 



-j^3 



iLix)i<i<m G Xd{g*+e* + (• I ri))(W). 
Hence, using [Sj Propositions 16.5(ii) and 16.8] and the notation (|2.ip . 

(0, ... ,0) G -(^X^^j I 5(r D^*) I ^ - E^iWJ 1 + X9(5* + A* + (• I n)Wi) 



(4.17) 



4 = 1 



[,Wl,...,U„ 



(4.18) 



In other words, by Fermat's rule, (ui,...,fm) solves (|4.2p . Finally, the strong convergence claims 
ii) (d) and (ii) (e) follow from Theorem I3.1|(ii)(e) \l (ii) (f ) since the uniform convexity of a function 



m 



ip G ro('H) at a point of the domain of 9(/7 implies the uniform monotonicity of dip at that point 
Section 3.4]. D 

In the following proposition we give conditions under which (j4.3p is satisfied. 

Proposition 4.3 Suppose that ()4.ip /las a^ least one solution and set 

S = {{LiX -yi)i<i<m I X G dom/ and {\fi G {1,. . . ,m}) y^ G dom^j + dom£j}. (4.19) 

Then ()4.3p is satisfied if one of the following holds. 

(i) (ri,...,rm) G sriS". 

(ii) For every i G {1, . . . , m}, gi or ii is real-valued. 
(iii) 7i and {Gi)i<i<m <ire finite- dimensional, and there exists x G ridom/ such that 

(Vi G {1, . . . ,?7i}) LjX — Tj G ridom /^i + ridom £j. (4.20) 



Proof. It follows from ()4.19p and [8", Proposition 12.6(ii)] that 

S = {{LiX - yi)i<i<m I X G dom/ and (Vi G {1, . . . ,m}) y^ G dom(yjn^j)} 

f m 

= < {LiX - yi)i<i<m X G dom(/ -\- h - {■ \ z)) and (yj)i<i<»n G X dom(fi(j Dj 



i=l 



X Li j ( dom (/ + /i - (• I z)) ) - dom 0(<7i D ^0 • 



(4.21) 
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M In view of (fOTD . 



(ri,...,rm) G sriS" 

(XLiVdom{f + h-{-\z))^-dom^igiDii)i--ri)y (4.22) 

Hence, since (X^i^j)* = 0[^ii.*, it follows from (gj]), dlJO]) . and [H Theorem 16.37(i)] that 

m m 

df + Y,mdgiDd£i){Li ■ -n) + wh-z = d{f + h-{-\z)) +Y,mdi9iaii)){Li ■ -n) 

i=l i=l 

= d(f + h-{-\z) + Y,{g^^(ii) o {Li ■ -ri)\ . (4.23) 
^ i=i ^ 

Since ()4.ip has at least one solution it follows from Fermat's rule that is in the range of the 
right-hand side of (|4.23|) . which shows that (|4.3|) holds. 



(ii) =^ (i) We have (Vi € {1, . . . , m}) dom^rj + dom^j = Qi. Therefore (|4.19p yields S = 0™ ^ ^*- 
(iii) =>(i) We have sriS = riS. However, it follows from (|4.2ip and [8, Corollary 6.15] that 

(XLMdom{f + h-{-\z))^ -dom^{g,Dii)j 

(m \ m 

XlA{ dom /) - ri dom (5, D ii) 

m \ m 

X Li ){h dom f) - X ndom{giD ii) 

i=l J i=\ 

m \ m 

X ij ) ( ri dom /) - X ( ri dom gi + ri dom ii) . (4.24) 

(=1 / i=l 



Hence (n, . . . ,rm) G sriS" 44> (3x G ridom/)(Vi G {1, . . . ,m}) LiX — ri G icidom. gi + ridom^j. D 

Remark 4.4 In Problem 14. H if each function ii is the indicator function of {0}, then (j4.ip reduces 
to 

m 

minimize f{x) + > gi{LiX — ri) + h{x) — {x \ z). (4-25) 

Even in this special case, the algorithm resulting from ()4.5p is new. This observation remains valid 
if we further assume that h: x 1— )• 0. 
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